Depth from Brightness of Moving Images

In this note we describe a method for recursively estimating the depth of a scene from a sequence of images. The input to the estimator are brightness values at a number of locations of a grid in a video image, and the output is the relative (scaled) depth corresponding to each image-point. The estimator is invariant with respect to the motion of the viewer, in the sense that the motion parameters are not part of the state of the estimator and therefore the estimates do not depend on motion as long as there is enough parallax (the translational velocity is nonzero). This scheme is a "direct" version of an other algorithm previously presented by the authors for estimating depth from point-feature correspondence independent of motion. Consider a sequence of images, consisting of a map from some location on a pixel grid x and a particular time instant t onto a brightness value in IR+. In practice the brightness values are quantized, and we will lump the effects of the quantization errors and other sensor noises into an additive Gaussian noise component, so that we measure I(x, t ) + ~ I ( x , t ) n~ E N(O, a) . (2) As the camera moves relative to the scene, the brightness patches on the image plane move accordingly. Under somewhat restrictive circumstances, we can assume that the brightness of each point in the scene remains unchanged. This assumption can be violated in a number *Research sponsored by NSF ERC in Neuromorphic Systems Engineering at Caltech, ONR grant N0001493-1-0990. This work is registered as CDS technical report n. CIT-CDS 95-008, March 1995.